This invention relates to expert systems. More specifically, this invention relates to translation of a neural network into a rule-based expert system. Further, this invention relates to a neural network which is trained as to better allow its translation.
The use of computers to store knowledge has been common for quite some time. In addition to simply storing data or information, computers have been used for drawing conclusions. For example, a computer may be used to predict the weather.
There has been a continuing need to develop computers or data processing systems which can interpret data or otherwise draw conclusions from data. The data processing system might draw the conclusions based upon numerous sample cases which have previously been supplied to it. For example, one might input current weather conditions and the data processing system or computer might predict the weather based upon the known weather patterns which occurred in the past. Instead of simply providing previous occurrences, one may want to supply rules to a computer or data processing system. The rules would tell the computer that when the facts corresponding to the premise of a rule occur, then the conclusion of the rule is true (or is most likely true).
Data processing systems have been developed which include neural networks. Such neural networks include layers of processing elements sometimes called neurons because they are designed to roughly simulate the neurons in a human brain. Various electrical circuits may be used to realize a processing element having an output which is a complex function of a series of inputs. In addition to the hardware neural networks, neural networks have been provided by software components in a digital computer system. The software components in such a software neural network simulate the hardware components of a hardware neural network. As used herein, a neural network will include both hardware and software neural networks and exclude neurons and connections in the brain of a human or animal. In other words, a neural network as used herein is artificial (non-human, non-animal).
Data processing systems have been used to establish, possibly through use of a machine learning program, rule-based expert systems. Such expert systems include a rule memory, often called a rule base, and an inference engine. Such systems determine which rules are applicable and then fire or execute the applicable rules in order to draw conclusions. Such systems have become increasingly popular because of many of their features. Generally, an expert system holding a given amount of knowledge would run significantly faster than a neural network holding the same amount of knowledge. Further, an expert system is structured in such a way that a human expert can follow the process by which the computer expert system reached its conclusion. The human expert can see which rules have been executed by the expert system and readily evaluate whether the expert system is functioning correctly. In contrast, it is very difficult for a human expert to follow the complex process used by a neural network, usually composed of one or more neural chips, because of the structure. A rule-based expert system is also advantageous relative to a neural network in that it is much easier usually to add a new rule to a rule-based expert system than it is to add further information to a neural network.
Although the features of a rule-based expert system as discussed above help explain the increasing popularity of such systems, such systems also have several disadvantages as compared to neural networks. Such expert systems search directly over the primary attribute space and this limits their search width. The primary attribute space refers to characteristics which are in the premises of the rules. If there are 100 different attributes or conditions, it may be difficult or impossible to search all possible combinations of the 100 attributes. In contrast, a neural network allows for an exponential search width because of its hidden layered structure. Thus, a neural network provides for an adequate search over practical problem domains. A further advantage of neural networks over expert systems is that neural networks may handle multivariate analysis, whereas rule-based expert systems are usually more limited in this aspect due to the relatively limited search width. A rule-based expert system can be established by a machine learning program in which one supplies specific cases such that these expert systems lack statistical generalization. In contrast, a neural network can handle statistical information relatively well and imply conclusions which are not addressed by a particular case supplied to it. A further advantage of neural networks is that they handle noisy data significantly better than an expert system as established by a symbolic learning program.